The present invention relates to an apparatus and a method of switching moving-picture bitstreams that have been coded to a small amount of codes by highly efficient coding, such as, MPEG-coding with inter-picture predictive coding for efficient video data transfer, storage and displaying.
Moving-picture bitstreams are switched in digital TV broadcasting or video data transfer, according to change in TV programs or content of data to be transferred. These systems require multiplexing (switching) of moving-picture bitstreams into one bitstream in edition. The edited bitstream requires high-fidelity decoding.
Pictures under MPEG-coding are divided into three different types I-, P- and B-pictures. I-pictures (intra-coded pictures) are coded independently, entirely without reference to other pictures. P-pictures (unidirectionally predictive-coded pictures) are compressed by coding the differences the pictures and reference preceding I- or P-pictures. B-pictures (bidirectionally predictive-coded pictures) are also compressed by coding the differences the pictures and reference preceding or upcoming I- or P-pictures. B-pictures are not used as reference pictures.
Inter-picture predictive coding requires an output buffer to follow a big difference in amount of codes generated for each picture. Coding is controlled so that a generated code amount becomes adequate for the buffer storage capacity, thus buffer occupancy being changed for each group of pictures.
Bitstream switching is performed so that the bitstream just after switching starts with an I-picture.
Switching of sequential two bitstreams generated by inter-picture predictive coding results in that the code amount overflowing or underflowing the buffer because the last buffer occupancy for the anterior bitstream and the initial buffer occupancy for the posterior bitstream at the switching timing are different from each other.
The difference in buffer occupancy is compensated for by inputting stuff data to increase an effective code amount or by inserting empty frames to decrease an effective code amount. The former technique does not affect reproduced pictures whereas the latter causes reproduction of no pictures for a period corresponding to empty frames.
Empty frames are mostly inserted at a bitstream-switching timing because the posterior bitstream starts with an I-picture of large amount of codes.
A well-known bitstream-switching apparatus switches MPEG-2 bitstreams xe2x80x9cAxe2x80x9d and xe2x80x9cBxe2x80x9d, using a blank inserter and a blank controller.
Suppose that each of the input bitstreams xe2x80x9cAxe2x80x9d and xe2x80x9cBxe2x80x9d carries 480 p(progressive)-pictures (720xc3x97480 pixels, about 60 frames per second) coded for every sixth-frame period of P(I)-pictures by MPEG-2 inter-picture predictive coding.
The input bitstreams xe2x80x9cAxe2x80x9d and xe2x80x9cBxe2x80x9d are supplied to the bitstream-switching apparatus and switched from the bitstream xe2x80x9cAxe2x80x9d to xe2x80x9cBxe2x80x9d at a predetermined timing. The switching timing is decided so that the anterior bitstream xe2x80x9cAxe2x80x9d is switched at the end of a P-picture to the posterior bitstream xe2x80x9cBxe2x80x9d starting with an I-picture, otherwise causing problems in inter-picture prediction for reproduction.
The switching is thus performed with detection of frame period and picture type to meet such requirement for switching timing, as illustrated in FIG. 1.
The output bitstream after switching has displacement of several frames at the switching timing, thus resulting in difference in buffer occupancy at the switching timing to cause overflowing or underflowing decoder buffer.
To avoid such difference in buffer occupancy, the output bitstream after switching is supplied to a blank inserter for inserting stuff bits or empty frames to the output bitstream in accordance with control data sent by a blank controller.
Stuff bits increase the number of bits for each predetermined reproduction period but will not be used in decoding. On the contrary, empty frames carrying no video data decrease the number of bits for each predetermined reproduction period. Insertion of empty frames causes delay in bitstream reproduction timing.
The amount of codes controlled by insertion of stuff bits or empty frames corresponds to the storage capacity of an output buffer. The storage capacity usually corresponds to the amount of bits to be transferred for about 0.2 to 0.5 seconds.
Therefore, insertion of stuff bits or empty frames sometimes causes no reproduction of pictures for about 0.2 to 0.5 seconds at maximum.
The bitstream processed by the blank inserter is supplied to an output buffer. The buffer stores the intermittently input bitstream and outputs it at a predetermined transfer bit rate. The buffer further detects change in amount of codes of the bitstream to send code amount data to the blank controller.
The bitstream after switching as described above is further supplied to a multiplexed-bitstream separator. The separator obtains buffer occupancy BO that is set as video buffering verifier VBV in MPEG, for each frame of the bitstream.
The BO data is supplied to the blank controller. Based on the BO data, the controller detects the displacement of frames at the switching timing. The controller sends control data in accordance with the displacement of frames to the blank inserter.
The inserter inserts stuff bits or empty frames to the bitstream after switching in accordance with the control data until the displacement of frames is cancelled, as illustrated in FIG. 2.
In FIG. 2, buffer occupancy BO is illustrated only at frame change. Buffer occupancy BO indicates decoder-buffer occupancy BO according to the MPEG standard. Thus, the larger the amount of codes for each frame, the lower the BO whereas the smaller the amount of codes for each frame, the higher the BO. The change in BO when raised becomes maximum with respect to transfer bit rate.
The bitstreams xe2x80x9cAxe2x80x9d and xe2x80x9cBxe2x80x9d have their own BO values which causes difference in buffer occupancy when switched according to the requirement for switching timing which is discussed above.
Insertion of empty frames to the bitstream at the switching timing promotes increase in buffer occupancy with respect to transfer bit rate due to no code amount for the empty frames.
Empty frames are inserted in unit of frame while stuff bits are inserted in unit of picture portion smaller than a frame so that there is a small difference in buffer occupancy as small as possible for the successive bitstreams xe2x80x9cAxe2x80x9d and xe2x80x9cBxe2x80x9d.
In FIG. 2, at the switching timing, the last buffer occupancy BO for the anterior bitstream xe2x80x9cAxe2x80x9d is indicated by the sign xe2x80x9caxe2x80x9d and the initial buffer occupancy BO for the posterior bitstream xe2x80x9cBxe2x80x9d is indicated by the sign xe2x80x9cbxe2x80x9d. The difference in buffer occupancy for the successive bitstreams xe2x80x9cAxe2x80x9d and xe2x80x9cBxe2x80x9d at the switching timing is expressed as xe2x80x9cb-axe2x80x9d.
Buffer occupancy for the bitstream xe2x80x9cBxe2x80x9d after switching becomes lower than xe2x80x9cbxe2x80x9d for the input bitstream xe2x80x9cBxe2x80x9d before switching by (b-a). In FIG. 2, buffer occupancy at a portion xe2x80x9ccxe2x80x9d of the input bitstream xe2x80x9cBxe2x80x9d is lowered to that at a portion xe2x80x9cdxe2x80x9d of the bitstream xe2x80x9cBxe2x80x9d after switching, thus underflowing the decoder buffer.
Several empty frames that correspond to the BO difference (b-a) are inserted into the output bitstream after switching to make less the BO difference.
Buffer occupancy at the portion xe2x80x9cdxe2x80x9d of the bitstream xe2x80x9cBxe2x80x9d before empty frame insertion is then raised to that at a portion xe2x80x9cexe2x80x9d of the bitstream xe2x80x9cBxe2x80x9d after empty frame insertion, or returned to that at the portion xe2x80x9ccxe2x80x9d, thus change in code amount before and after switching becoming small for the storage capacity of the decoder buffer.
Insertion of empty frames for making small the difference in buffer occupancy at the switching timing, however, causes delay in reproduction start timing for the posterior bitstream xe2x80x9cBxe2x80x9d.
Such delay results in no pictures to be decoded after the switching timing for a period that corresponds to the inserted empty frames, thus causing discontinuity in moving picture when reproduced.
A purpose of the present invention is to provide an apparatus and a method of switching moving-picture bitstreams with no discontinuity in moving picture after switching.
The present invention provides a bitstream switching apparatus for switching a plurality of incoming moving-picture bitstreams each carrying first pictures to be used as reference pictures for inter-picture prediction and second pictures other than the first pictures, at a predetermined switching timing, to form a successive moving-picture bitstream, the apparatus including: a data separator to obtain buffer occupancy indicator at decoder buffer on each picture carried by the successive moving-picture bitstream; a decimation controller to obtain a difference in buffer occupancy just before and just after the switching timing based on the buffer occupancy data to set a decimation rate and a decimation period on the second pictures so that the difference in buffer occupancy just before and just after the switching timing becomes small; and bitstream decimator to decimate the second pictures only from the successive moving-picture bitstream at the decimation rate within the decimation period, thus obtaining an output moving-picture bitstream having an adjusted buffer occupancy in accordance with the decimation.
Moreover, the present invention provides a bitstream switching method for switching a plurality of incoming moving-picture bitstreams each carrying first pictures to be used as reference pictures for inter-picture prediction and second pictures other than the first pictures, at a predetermined switching timing, to form a successive moving-picture bitstream. Buffer occupancy data is obtained on each picture carried by the successive moving-picture bitstream. A difference in buffer occupancy just before and just after the switching timing is obtained based on the buffer occupancy data to set a decimation rate and a decimation period on the second pictures so that the difference in buffer occupancy just before and just after the switching timing becomes small with respect to storage capacity of a buffer for storing the successive moving-picture bitstream. The second pictures only are decimated from the successive moving-picture bitstream at the decimation rate within the decimation period, thus obtaining an output moving-picture bitstream having buffer occupancy adequate for the storage capacity of the buffer.